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Wow, The Web is BIG 


Updated November 10, 1994! Really big. Ok, let's look at it this way. My wanderer takes forever 
to run now and I don't run it very frequently partially because of this. At last check, it had found 
870 sites, and that's mostly only because I didn't let it finish. Let's look at the traffic over the NSF. 
Backbone. Specifically, let's look at three specific quantities. 

e The total amount of data that went over the web in 1992 

¢ The total amount of data that went over the web in Jan-Mar of 1993 

¢ The total amount of data that went over the web on May 1, 1994 

¢ The total amount of data that went over the web in the last 6 hours 
Guess which one is biggest. Guess which one is biggest by a lot. If you guessed the first or the 
second, you're wrong. 


The total for 1992 is about SOOM. That's not a small amount of data, and remember, all data that 
goes over the web within a certain area never touches the backbone, so doesn't get counted in this 
figure. Actually, that's a lot of data. About 500,000 pages of text or 1000 fairly large images. 
Though, on physical disk, I can easily carry that much disk in one hand. 


The total for Jan-Mar of 1993 is about 5G. That's 10 times as much as all of 1992. That's a lot of 
data. If that were disk space it would actually cost quite a bit, (about $5,000, I think) for me at 
least. It's a lot, really. On physical disk, it's a few disks, but once again, carryable, though one arm 
may be pushing it. 


The total for one day in May Is approximately 10G. Wow. One day. I don't want to think about it. 
The amount that went over the backbone in one month, that is, February 1994 is 347G. 347 
Gigabytes of data! That's a hell of a lot. Ok, so 11.4 Terabytes go over the whole net, and that's 
more than a hell of a lot. That's so much that if you really try and conceive of how big it is, your 
brain will coredump and you'll be fsck'ing childhood memories for hours. Don't think about it. 
And physical disk. Heh. I've seen 40 or 50 gig of disk in one place and that's a lot. As much as 
269 gig in one place is just, well, a lot. 


The total for the last 6 hours (for September 1994) is about 13G. Aaaaaah! Let's do some math. 
How much data went over the web in 1992? How much data went over the web in the last 15 


minutes? About the same amount. Actually, more probably went over in the last 15 minutes. Not 
kidding here. 


This all corresponds to more than 200,000% annual growth. Ok, so that's exaggerating to include 
data that far back. Let's just look at the past few months or so. Ooh, that lowers the value a lot. 
Down to a diminutive 3000% percent annually. That's a 1% growth per day. In case it's not 
apparent, I’m impressed. The SIPB Web Server , the one I maintain, has seen lots of usage. 
extrapolating our data for the past year. By Christmas of '94, we should be getting 1000 
connections per second during peak usage. We may have to get a faster machine. Isn't extrapolation 
fun boys and girls :-) Ok, so that's an exaggeration. A little. At the end of October, we were getting 
about | connection per second, average. Six months prior, we were getting about 1 connection 
every 4 or 5 seconds. Peak usage is about 2.5 times average, so we have another couple years 
before we're at 1000 connections per second. Oh well. It was a fun extrapolation while it lasted. In 


any case, we can still predict that there will be one web server for every man, woman and child on 
the planet before the turn of the millenium. (actually, sometime in 1998 :-) 


Really though, clearly a lot of this is the initial growth of the web, but it really is growing really 
really really fast. Ok. Enough of me. :-) 
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